Mobile Price Prediction using Regression Techniques¶
1. Import the necessary Libraries¶
In [64]:
import pandas as pd
import numpy as np
import seaborn as sns
from matplotlib import pyplot as plt
2. Load and About Dataset¶
In [65]:
df = pd.read_csv("data/data.csv")
In [66]:
df.head()
Out[66]:
Independent Features¶
- Unnamed: 0 - Index column and this is an unnecessary column
- Brand me — Brand of the mobile phone and the model.
- Ratings — Consumer rating for the mobile phone
- RAM — RAM size of the mobile
- ROM — ROM (Internal Memory) size of the mobile
- Mobile_Size — Size of the mobile in inches
- Primary_Cam — Pixel size of the back camera
- Selfi_Cam — Pixel size of the front/selfi camera
- Battery_Power — Battery capacity of the mobilr in mAh
Dependent Features¶
- Price — Price of the mobile
In [264]:
df.shape
Out[264]:
In [265]:
df.info()
3. Exploratory Data Analysis(EDA)¶
In [267]:
plt.figure(figsize=(10,10))
sns.pairplot(data=df)
Out[267]:
If you need to do further analysis about the data, please follow the Auto EDA library (Pandas Profiling)
In [4]:
pip install pandas-profiling
In [6]:
#importing required packages
import pandas as pd
import pandas_profiling
import numpy as np
#descriptive statistics
profile = pandas_profiling.ProfileReport(df)
In [7]:
profile
Out[7]:
In [9]:
profile.to_file("EDA_report.html")
In [273]:
df.describe()
Out[273]:
4. Data Preprocessing¶
4.1 Drop unnecessary columns¶
We can see Unnamed: 0 column is useless because it doesn't give any meaningful thing and it's just an index column. Therefore we can remove that column from the dataset.
In [40]:
df.drop(['Unnamed: 0'], axis = 1, inplace = True)
In [41]:
df.columns
Out[41]:
4.2 Dealing with missing values¶
To check the number of missing values in the dataset,
In [307]:
df.isnull().sum()
Out[307]:
In [105]:
# To visulize it
sns.heatmap(df.isnull(),yticklabels=False,cbar=False,cmap='viridis')
plt.show()
We can see Ratings, RAM, ROM, Mobile_Size, Selfi_Cam variables have missing values and we need to treat them first.
In here we will apply bellow techniques to deal with missing values and among them we will find what is the most appropriate missing value technique for each variable.
- Mean/Medain/Mode Imputation
- Random number Imputation
4.2.1 Mean/Medain/Mode Imputation¶
When the data follow normal distribution, mean value replacement is the best method. But when the data skewed it's better to do the meadian value replacement.
In [42]:
missing_var = ['Ratings', 'RAM', 'ROM', 'Mobile_Size', 'Selfi_Cam']
Put all the missing value variables into an another dataframe and name it as df_missing.
In [43]:
df_missing= df[['Ratings', 'RAM', 'ROM', 'Mobile_Size', 'Selfi_Cam']]
df_missing.head()
Out[43]:
To look at the distributions of the variables which has the missing values.
In [108]:
plt.figure(figsize=(5,5))
sns.pairplot(data=df_missing)
Out[108]:
By looking at the above graph's diagonal(histograms), it shows all the varbales are skewed. Therefore we will use median imputation technique.
In [44]:
# If the variable has a missing value, this function will repalce that missing value by it's median
def impute_nan(df,variable):
df[variable+"_median"]=df[variable].fillna(df[variable].median())
In [45]:
# run above function for all the missing value variable
for i in missing_var:
impute_nan(df_missing,i)
In [51]:
df_missing.head()
Out[51]:
In [52]:
df_missing.isnull().sum()
Out[52]:
4.2.2 Random number Imputation¶
In [46]:
# If the variable has a missing value, this function will repalce that missing value by a random number (this random number will choose from the relavant variable)
def impute_nan_random(df,variable):
df[variable+"_random"]=df[variable]
##It will have the random sample to fill the na
random_sample=df[variable].dropna().sample(df[variable].isnull().sum(),random_state=0)
##pandas need to have same index in order to merge the dataset
random_sample.index=df[df[variable].isnull()].index
df.loc[df[variable].isnull(),variable+'_random']=random_sample
In [47]:
for i in missing_var:
impute_nan_random(df_missing,i)
In [55]:
df_missing.head()
Out[55]:
In [56]:
df_missing.isnull().sum()
Out[56]:
In [117]:
#Comparison of the distrbutions before and after deali with missing values
def comparison_plot(df, variable):
fig = plt.figure()
ax = fig.add_subplot(111)
df[variable].plot(kind='kde', ax=ax)
df[variable+"_median"].plot(kind='kde', ax=ax, color='red')
df[variable+"_random"].plot(kind='kde', ax=ax, color='green')
lines, labels = ax.get_legend_handles_labels()
ax.legend(lines, labels, loc='best')
In [118]:
for i in missing_var:
comparison_plot(df_missing,i)
In [119]:
df.columns[0]
Out[119]:
In [57]:
#Compare the standard values before and after imputation
def compare_std(df, variable):
print("original std of ", variable, "is ", df[variable].std(),
"and after median repalcement std: ", df[variable+"_median"].std(),
"and after random number repalcement std: ", df[variable+"_random"].std())
In [58]:
for i in missing_var:
compare_std(df_missing,i)
By looking at the kernel density plots and the standard values, we can obtained random number imputation method is the best method as it doesn't distrote the original distribution's behaviors.
In [122]:
#Combine df_missing and df dataframes
df_missing.head()
Out[122]:
In [123]:
df_missing.columns
Out[123]:
In [48]:
df_new = pd.concat([df, df_missing[['Ratings_random', 'RAM_random', 'ROM_random', 'Mobile_Size_random',
'Selfi_Cam_random']]], axis = 1)
In [60]:
df_new.head()
Out[60]:
In [49]:
#remove the unnecessary columns
df_new.drop(['Ratings', 'RAM', 'ROM', 'Mobile_Size','Selfi_Cam'], axis = 1, inplace = True)
In [62]:
df_new.head()
Out[62]:
In [26]:
df_new.isnull().sum()
Out[26]:
Now we have a clean dataset
4.3 Modify Variables(Feature Engineering)¶
In [50]:
df_new['Brand me'].dtype
Out[50]:
In [51]:
df_new['Brand me'] = df_new['Brand me'].str.split(' ',1).str[0]
In [52]:
df_new['Brand me'].value_counts()
Out[52]:
In [66]:
df_new.head()
Out[66]:
In [15]:
import seaborn as sns
In [16]:
plt.figure(figsize=(15,10))
sns.set_style('whitegrid')
sns.countplot(x='Brand me',
data=df_new,
order = df_new['Brand me'].value_counts().index)
plt.xticks(rotation=75)
plt.show()
In this chart we can see, there are 71 mobile products in this datset and there are only few companies has sales more than 20. Therefore we select only the top 10 brands in the dataset and we rename other variables as Other category.
In [53]:
#Get the list of top 10 elements
top10 = df_new['Brand me'].value_counts().index[:10]
top10
Out[53]:
In [54]:
df_new['Brand me'] = np.where(df_new['Brand me'].isin(top10), df_new['Brand me'], 'Other')
df_new['Brand me'].nunique()
Out[54]:
In [71]:
df_new['Brand me'].value_counts()
Out[71]:
Now we have a categorical feature and it has 11 categories. Therefore we need to encode them.
4.4 Handling Categorical Data¶
In [55]:
brand_list = df_new['Price'].groupby(df_new['Brand me']).mean().sort_values(ascending=False)
brand_list
Out[55]:
We an see there's a relationship between price and the brand. Therefore we will use nominal encoding method to encode the Brand me variable.
In [20]:
#get the list
brand_list.index
Out[20]:
In [21]:
brand_list.nunique()
Out[21]:
In [56]:
#First convert to Categorical
df_new['Brand me'] = df_new['Brand me'].astype('category')
#rename the columns using a dictionary
df_new['Brand me'] = df_new['Brand me'].cat.rename_categories({'Apple': 11,
'Samsung': 10,
'Vivo': 9,
'OPPO': 8,
'Other': 7,
'Nokia': 6,
'Lava': 5,
'Micax': 4,
'Karbonn': 3,
'I': 2,
'Kechaoda':1
})
In [76]:
df_new['Brand me'].value_counts()
Out[76]:
In [77]:
df_new.head()
Out[77]:
In [78]:
df_new.shape
Out[78]:
4.5 Feature Selection¶
Understanding the correlation of features between target and other features.
In [238]:
corr = df_new.corr()
corr.shape
Out[238]:
In [247]:
# Plotting the heatmap of correlation between features
plt.figure(figsize=(10,10))
sns.heatmap(corr, cbar=False, square= True, annot=True, cmap='Blues')
Out[247]:
By looking at the above heatmap there's no multicollinearity in the dataset. You can confirm it by using the bellow function.
In [248]:
# with the following function we can select highly correlated features
# it will remove the first feature that is correlated with anything other feature
def correlation(dataset, threshold):
col_corr = set() # Set of all the names of correlated columns
corr_matrix = dataset.corr()
for i in range(len(corr_matrix.columns)):
for j in range(i):
if abs(corr_matrix.iloc[i, j]) > threshold: # we are interested in absolute coeff value
colname = corr_matrix.columns[i] # getting the name of column
col_corr.add(colname)
return col_corr
In [249]:
corr_features = correlation(df_new, 0.7)
len(set(corr_features))
Out[249]:
This function also implies, there's no inter correlation between independent variables.
4.6 Feature Importance¶
In [250]:
df_new.head()
Out[250]:
In [251]:
df_new.columns
Out[251]:
In [57]:
X = df_new[['Brand me', 'Primary_Cam', 'Battery_Power', 'Ratings_random',
'RAM_random', 'ROM_random', 'Mobile_Size_random', 'Selfi_Cam_random']]
y = df_new['Price']
In [24]:
# Fiting Feature Seclection using Ensemble Methods
from sklearn.ensemble import ExtraTreesClassifier
import matplotlib.pyplot as plt
model = ExtraTreesClassifier()
model.fit(X,y)
Out[24]:
In [256]:
print(model.feature_importances_)
In [259]:
# Plot graph of feature importances for better visualization
feat_importances = pd.Series(model.feature_importances_, index=X.columns)
feat_importances.nlargest(10).plot(kind='barh')
plt.show()
4.7 Dealing with outliers¶
In [274]:
sns.distplot(df_new['ROM_random'])
Out[274]:
In [275]:
figure=df_new.boxplot(column="ROM_random")
In [276]:
df_new['ROM_random'].describe()
Out[276]:
In [277]:
df_new.loc[df_new['ROM_random']>64, :]
Out[277]:
In [280]:
outliers = df_new.loc[df_new['ROM_random']>64, :]
outliers['Brand me'].value_counts()
Out[280]:
We can see 7-11 categories have the outliers and it may be can occure becuase their avergae prices are higher than the others. So usually when the prices are increasing, features also increasing. TBy looking at the above stats it is unfair to replace them with other values. So we don't touch with these outliers as it give us very usefull information.
4.8 Transform the data¶
We can clearly see the scales of the variables are very different. Therefore we need to apply a transformation technique at the end.
In [58]:
#### standarisation: We use the Standardscaler from sklearn library
from sklearn.preprocessing import StandardScaler
scaler=StandardScaler()
### fit vs fit_transform
X=scaler.fit_transform(X)
In [81]:
X
Out[81]:
5. Model Fitting¶
5.1 Split the dataset¶
In [59]:
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.3)
5.2 Linear Regression¶
In [60]:
from sklearn import linear_model
from sklearn.metrics import mean_squared_error, r2_score
In [98]:
# Create linear regression object
regr = linear_model.LinearRegression()
# Train the model using the training sets
regr.fit(X_train, y_train)
Out[98]:
In [99]:
# Make predictions using the testing set
y_pred_lr = regr.predict(X_test)
In [105]:
sns.histplot(y_test-y_pred_lr)
plt.show()
In [106]:
# Visualizing the differences between actual prices and predicted values
plt.scatter(y_test, y_pred_lr)
plt.xlabel("Prices")
plt.ylabel("Predicted prices")
plt.title("Prices vs Predicted prices")
plt.show()
In [107]:
# The coefficients
print("Coefficients: \n", regr.coef_)
# The mean squared error
print("Mean squared error: %.2f" % mean_squared_error(y_test, y_pred_lr))
# The coefficient of determination: 1 is perfect prediction
print("Coefficient of determination: %.2f" % r2_score(y_test, y_pred_lr))
In [108]:
# Training Score
print("Training Accuracy:",regr.score(X_train,y_train)*100)
# Testing Score
print("Testing Accuracy:",regr.score(X_test,y_test)*100)
We can see accuracies are really poor. Therefore linear regression is not a good model.
5.3 Elastic Net Regression¶
In [109]:
from sklearn.linear_model import ElasticNet
In [110]:
EN_model = ElasticNet(alpha=1.0, l1_ratio=0.5)
In [112]:
EN_model.fit(X_train,y_train)
Out[112]:
In [113]:
# Training Score
print("Training Accuracy:",EN_model.score(X_train,y_train)*100)
In [114]:
# Testing Score
print("Testing Accuracy:",EN_model.score(X_test,y_test)*100)
5.4 ExtraTreesClassifier¶
In [84]:
from sklearn.ensemble import ExtraTreesClassifier
In [85]:
model = ExtraTreesClassifier()
model.fit(X_train,y_train)
Out[85]:
In [86]:
y_pred_etc = model.predict(X_test)
In [90]:
sns.histplot(y_test-y_pred_etc)
plt.show()
In [91]:
plt.scatter(y_test, y_pred_etc, alpha = 0.5)
plt.xlabel("y_test")
plt.ylabel("y_pred")
plt.show()
In [93]:
from sklearn import metrics
In [94]:
print('MAE:', metrics.mean_absolute_error(y_test, y_pred_etc))
print('MSE:', metrics.mean_squared_error(y_test, y_pred_etc))
print('RMSE:', np.sqrt(metrics.mean_squared_error(y_test, y_pred_etc)))
In [95]:
# RMSE/(max(DV)-min(DV))
35630.175346287186/(max(y)-min(y))
Out[95]:
In [87]:
# Training Score
print("Training Accuracy:",model.score(X_train,y_train)*100)
In [88]:
# Testing Score
print("Testing Accuracy:",model.score(X_test,y_test)*100)
We can see there's an overfitting issue in this model as it has massive difference between training accuracy and testing accuracy.
5.5 Random Forest Regressor¶
In [119]:
from sklearn.ensemble import RandomForestRegressor
reg = RandomForestRegressor()
reg.fit(X_train,y_train)
Out[119]:
In [ ]:
from sklearn.ensemble import RandomForestRegressor
reg = RandomForestRegressor()
reg.fit(X_train,y_train)
In [120]:
y_pred = reg.predict(X_test)
In [359]:
# Visualizing the differences between actual prices and predicted values
plt.scatter(y_test, y_pred)
plt.xlabel("Prices")
plt.ylabel("Predicted prices")
plt.title("Prices vs Predicted prices")
plt.show()
In [124]:
# Training Score
print("Training Accuracy:",reg.score(X_train,y_train)*100)
In [123]:
# Testing Score
print("Testing Accuracy:",reg.score(X_test,y_test)*100)
In [125]:
print('MAE:', metrics.mean_absolute_error(y_test, y_pred))
print('MSE:', metrics.mean_squared_error(y_test, y_pred))
print('RMSE:', np.sqrt(metrics.mean_squared_error(y_test, y_pred)))
This is a good model. Therefore we will optimize this model to improve the performances.
5.6 Support Vector Regressor¶
In [354]:
from sklearn.svm import SVR
regressor = SVR(kernel = 'rbf')
regressor.fit(X_train, y_train)
Out[354]:
In [360]:
y_pred = reg.predict(X_test)
In [365]:
# Training Score
print("Training Accuracy:",regressor.score(X_train,y_train)*100)
In [366]:
# Testing Score
print("Testing Accuracy:",regressor.score(X_test,y_test)*100)
6. Hyper-Parameter Tuning¶
6.1 Randomized Search¶
In [126]:
from sklearn.model_selection import RandomizedSearchCV
In [127]:
#Randomized Search CV
# Number of trees in random forest
n_estimators = [int(x) for x in np.linspace(start = 100, stop = 1200, num = 12)]
# Number of features to consider at every split
max_features = ['auto', 'sqrt']
# Maximum number of levels in tree
max_depth = [int(x) for x in np.linspace(5, 30, num = 6)]
# Minimum number of samples required to split a node
min_samples_split = [2, 5, 10, 15, 100]
# Minimum number of samples required at each leaf node
min_samples_leaf = [1, 2, 5, 10]
# Create the random grid
random_grid = {'n_estimators': n_estimators,
'max_features': max_features,
'max_depth': max_depth,
'min_samples_split': min_samples_split,
'min_samples_leaf': min_samples_leaf}
In [128]:
# Random search of parameters, using 3 fold cross validation,
# search across 100 different combinations
rf_random = RandomizedSearchCV(estimator = reg, param_distributions = random_grid,scoring='neg_mean_squared_error', n_iter = 10, cv = 3, verbose=2, random_state=42, n_jobs = 1)
rf_random.fit(X_train,y_train)
Out[128]:
In [129]:
rf_random.best_params_
Out[129]:
In [132]:
best_random_grid=rf_random.best_estimator_
In [ ]:
prediction = best_random_grid.predict(X_test)
In [ ]:
plt.figure(figsize = (8,8))
sns.distplot(y_test-prediction)
plt.show()
In [135]:
plt.figure(figsize = (8,8))
plt.scatter(y_test, prediction, alpha = 0.5)
plt.xlabel("y_test")
plt.ylabel("y_pred")
plt.show()
In [136]:
print('MAE:', metrics.mean_absolute_error(y_test, prediction))
print('MSE:', metrics.mean_squared_error(y_test, prediction))
print('RMSE:', np.sqrt(metrics.mean_squared_error(y_test, prediction)))
In [137]:
# Training Score
print("Training Accuracy:",best_random_grid.score(X_train,y_train)*100)
# Testing Score
print("Testing Accuracy:",best_random_grid.score(X_test,y_test)*100)
6.2 Grid Search¶
In [138]:
from sklearn.model_selection import GridSearchCV
param_grid = {
'n_estimators': [rf_random.best_params_['n_estimators']+50,
rf_random.best_params_['n_estimators']-50],
'min_samples_split': [rf_random.best_params_['min_samples_split']+1,
rf_random.best_params_['min_samples_split']+2,
rf_random.best_params_['min_samples_split']+3],
'min_samples_leaf': [rf_random.best_params_['min_samples_leaf']],
'max_features': [rf_random.best_params_['max_features']],
'max_depth': [rf_random.best_params_['max_depth'] - 2,
rf_random.best_params_['max_depth'] - 1,
rf_random.best_params_['max_depth'],
rf_random.best_params_['max_depth'] +1,
rf_random.best_params_['max_depth'] + 2]
}
print(param_grid)
In [139]:
#### Fit the grid_search to the data
grid_search=GridSearchCV(estimator=reg,param_grid=param_grid,cv=5,n_jobs=-1,verbose=2)
grid_search.fit(X_train,y_train)
Out[139]:
In [140]:
grid_search.best_estimator_
Out[140]:
In [141]:
best_grid=grid_search.best_estimator_
In [142]:
best_grid
Out[142]:
In [143]:
# Training Score
print("Training Accuracy:",best_grid.score(X_train,y_train)*100)
# Testing Score
print("Testing Accuracy:",best_grid.score(X_test,y_test)*100)
It reduce the both accuracies. Therefore we will go with the randomized search method's model as a best model.
7. Save the best model to reuse it again¶
In [144]:
import pickle
# open a file, where you ant to store the data
file = open('mobile_price_rg.pkl', 'wb')
# dump information to that file
pickle.dump(best_random_grid, file)
In [146]:
#Load the model
model = open('mobile_price_rg.pkl','rb')
reg_rand = pickle.load(model)
In [147]:
y_prediction = reg_rand.predict(X_test)
In [148]:
metrics.r2_score(y_test, y_prediction)
Out[148]:
In [ ]: